Face Recognition From Video

While face recognition (FR) from a single still image has been studied extensively [13], [57], FR based on a video sequence is an emerging topic, evidenced by the growing increase in the literature. It is predictable that with the ubiquity of video sequences, FR based on video sequences will become more and more popular. In this chapter, we also address FR based on a group of still images (also referred to as multiple still images). Multiple still images are not necessarily from a video sequence; they can come from multiple independent still captures. It is obvious that multiple still images or a video sequence can be regarded as a single still image in a degenerate manner. More specifically, suppose that we have a group of face images {y1, . . . ,yT } and a single-still-image-based FR algorithm A (or the base algorithm), we can construct a recognition algorithm based on multiple still images or a video sequence by fusing multiple base algorithms denoted by Ai’s. Each Ai takes a different single image yi as input. The fusion rule can be additive, multiplicative, and so on. Even though the fusion algorithm might work well in practice, clearly, the overall recognition performance solely depends on the base algorithm and hence designing the base algorithm A (or the similarity function k) is of ultimate importance. However, the fused algorithms neglect additional properties manifested in multiple still images or video sequences. Generally speaking, algorithms that judiciously exploit these properties will perform better in terms of recognition accuracy, computational efficiency, etc. There are three additional properties available from multiple still images and/or video sequences: [P1: Set of observations]. This property is directly exploited by the fused algorithms. One main disadvantage may be the ad hoc nature of the combination rule. However, theoretical analysis based on a set of observations can be performed. For example, a set of observations can be summarized using quantities like matrix, probability density function, manifold, etc. Hence, corresponding knowledge can be utilized to match two sets. [P2: Temporal continuity/Dynamics]. Successive frames in the video sequences are continuous in the temporal dimension. Such continuity, coming from facial expression, geometric continuity related to head